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Abstract. 

The statistical properties of the energy landscape of the low autocorrelated binary 
string problem (LABSP) are studied numerically and compared with those of several 
classic disordered models. Using two global measures of landscape structure which have 
been introduced in the Simulated Annealing literature, namely, depth and difhculty, 
we find that the landscape of LABSP, except perhaps for a very large degeneracy of the 
local minima energies, is qualitatively similar to some well-known landscapes such 
as that of the mean-field 2-spin glass model. Furthermore, we consider a mean- 
field approximation to the pure model proposed by Bouchaud and Mezard (1994, J. 
Physique I France 4 1109) and show both analytically and numerically that it describes 
extremely well the statistical properties of LABSP. 
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1. Introduction 

The Low Autocorrelated Binary String Problem (LABSP) p5| , Q consists of finding binary 
strings x of length N over the alphabet {±1} with low aperiodic off-peak autocorrelation 
Rk{x) = J^iLi' ^i^i+k for all lags k. These strings have technical applications such as the 
synchronization in digital communication systems and the modulation of radar pulses. 
The quality of a string x is measured by the fitness or energy function 
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In most of the literature on the LABSP the merit factor F{x) = N"^ / {AT-[{x)) is used (see 
e.g. 0): using TC instead is more convenient for explicit computations. 

Recently there has been much interest in frustrated models without explicit 
disorder. The LABSP and related bit-string problems have served as model systems 
for this avenue of research [p!9| , [20| , |23| , |^ . These investigations have lead to a claim that 
LABSP has a 'golf-course' type landscape structure, which would explain the fact that it 
has been identified as a particularly hard optimization problem for heuristic algorithms 
such as Simulated Annealing (see ^ and the references therein). 

The landscape of LABSP consists of a (dominant) 4-spin Hamiltonian plus an 
asymptotically negligible quadratic component. We note that the generic 4-spin 
landscape is Derrida's 4-spin Hamiltonian |TT| which is a linear combination of all (^) 



distinct 4-spin functions, while the LABSP Hamiltonian, on the other hand, only contains 
(9(A^^) non- vanishing 4-spin contributions. The landscape of the LABSP thus corresponds 
to a dilute 4-spin ferro magnet. Numerical simulations in show that the LABSP has by 



far more local optima than a generic 4-spin glass model, which corroborates the rather 
surprising finding that disordered ferromagnets have more metastable states than their 
spin-glass counterparts p. 

In this contribution we carry out a thorough investigation of the statistical 
properties of the energy landscape of LABSP aiming at to determine whether it has 
any peculiar features that would lead to a 'golf-course' structure, with vanishingly 
small correlations between the energies of neighboring states. To do so we carry out a 
comparison with four disordered models, namely, the random energy model (REM) |[TT[1 , 
the ±1 4-spin glass model a mean-field approximation to H (MF) |Q, which 

reproduces the results of Golay's ergodicity assumption [|1^], and, finally, the ±1 2-spin 



glass model [Q. The replica analyses indicate that the first three models have a rather 
unusual spin-glass phase, where the overlap between any pair of different equilibrium 
states vanishes, while the last model has a normal spin-glass phase described by a 
continuous order parameter function. 

The rest of this paper is organized in the following way. In section |^ we 
calculate analytically the average density of local minima of the disordered mean- 
field approximation to 7i and show that it indeed describes very well the statistics 
of metastable states of the pure model. Rather surprisingly, we find that the value of 
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the energy density at which the density of local minima vanishes coincides with the 
bound predicted by Golay [|1^], as well as with the ground-state energy predicted by 
the first step of replica-symmetry breaking calculations of the mean-field model [Q. To 
properly compare the landscapes of the different models mentioned above, in section ^ 
we consider two global measures of landscape structure which have been introduced in 
the Simulated Annealing literature: depth and difficulty ||T^, |TH|, We show that 
LABSP, the mean-field approximation, and the binary ±1 2 and 4-spin glasses exhibit 
approximately the same qualitative behavior in these parameters, while the behavior 
pattern of the random energy model departs significantly from those. Finally, in section 
§ we summarize our main results and present some concluding remarks. 



2. Mean- field approximation 



Bouchaud and Mezard and, independently, Marinari et al. [|19[ have proposed the 
following disordered model, which is "as close as possible" to the pure model: 
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Here the coupling strengths Jf^ ^ Jj^ are statistically independent random variables that 
can take on the value 1 with probability (A^ — k) /N"^ and zero otherwise. Hence the 
average number of bonds in 7i and Tid is the same, namely, N — k. Moreover, the pure 
model is recovered with the choice J^j = Si^kj- Probably the most appealing feature 
of this model is that its high-temperature (replica-symmetric) free-energy is identical 
to that obtained by Bernasconi |^ using Golay's ergodicity assumption [jl5l, in which 



the squared autocorrelations Rl are treated as independent random variables. As the 
constraints of the one-dimensional geometry are lost in the disordered Hamiltonian Tid, 
it can be viewed as the mean field version of Ti. 

The thermodynamics of the disordered model is interesting on its own since, 
similarly to the random energy model [|l^ , it presents a first order transition at a certain 
temperature Tg, below which the overlap between any pair of different equilibrium states 
vanishes 0]. In contrast to the random energy model, however, the degrees of freedom 
are not completely frozen for T < Tg, and the entropy vanishes linearly with T as the 
temperature decreases towards zero. To better understand the low-temperature phase 
of the mean-field Hamiltonian Tid, in the following we will calculate analytically the 
expected number of metastable states {Af (e)) with a given energy density e. 

The energy cost per site of flipping the spin Xi is 57i^ = — Aj where 



with 
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We say that a state x = {xi, . . . ,xi\f) is a strict local minimum if Aj < for all i; 
in the case that the equality Aj = holds for some i, we call x a degenerate local 
minimum. In the forthcoming analysis, the choice of < instead of <, which is customary 
in optimization theory, see e.g. does not make any difference. In section ^, however, 
degeneracies will play a role. 

The average number of local minima with energy density e can be written as 



(A^(e)) 



Tr .x 6 
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(5) 



where Tr^. denotes the summation over the 2^ spin configurations and (...) stands for 
the average over the couplings J^j. Here 6 (x) = 1 if x > and otherwise, and 6 (x) 
is the Dirac delta-function. 

Using the integral representation of the delta-function we obtain 
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The average over the couplings can be easily carried out and, in the thermodynamic 
limit — >■ cx), it yields 



ln(. . .) 



k 




(7) 



We note that this result could have been obtained by considering the couplings 
as Gaussian independent random variables with means and variances equal to 
(1 — k/N) /N. To get a physical but nontrivial thermodynamic limit we must assume 
that the magnetization Xi scales with A^^/^, which results then in the vanishing of the 
term that contains the dependence on the spin variables in eq.(|^). Droping this term, 
the sum over the spin configurations yields simply 2^. As the remaining calculations 
are rather straightforward we will only sketch them in the sequel. 

To carry out the integrals over and we introduce the auxiliary parameters 
^Qk = '^ii'Vi) , Nmk = J2i^i^ Tfc = J2i'^i- After performing the resulting 
Gaussian integrals we introduce the saddle-point parameters NM = - Aj and NQ = 
^ . A^ which allow the decoupling of the indices k and i. The final result is 
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The integrals in eq.(|D are then evaluated in the hmit oo by the standard saddle- 
point method, while the integrals in the equations for Gq and Gi are trivially performed. 
The final result for the exponent 

" ^ln(Ar(e)) (11) 
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where the saddle-point parameters M, M, Q, Q-, and e are determined so as to maximize 
a. In particular, a brief analysis of the saddle-point equations indicates that M, Q and 
e are imaginary so that a is real, as expected. Introducing the real parameters = iM, 

P = ie,ri = M/ UiQj , and ^ = -Q/M'^, we rewrite eq.(|l2D as 



a (e) = 2/i — — + /3e + In erfc (rj) 



1 /"^ 
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where we have used the saddle-point equation da/dQ = to eliminate Q. We note that 
in eq. (|13]) the parameters 77 and are decoupled which facilitates greatly the numerical 
problem of maximizing a. 

The number of local minima, regardless of their particular energy values, is obtained 
by maximizing a with respect to e, which corresponds to setting /3 = in the saddle- 
point equations. In this case, the value of the energy density that maximizes a, denoted 
by et, can be interpreted as the typical (average) energy density of the local minima. 
We find a = 0.4394 and = 0.0837. These results agree very well with the numerical 
data a ~ 0.4388 ± (7) and ~ 0.0826 ± (6), obtained through the exhaustive search for 
< 20 and averaging over 100 realizations of the couplings. 

Moreover, an exhaustive search for < 30 yields that the exponent governing the 
exponential growth of the number of local minima in the pure model 7i is 0.453 it (7) and 
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Figure 1. Exponent a{e) as a function of the energy density e. The ground state 
energy is defined by a(eo) = 0. We find eq = 0.0202845. 



the typical energy density of the minima is 0.086 ± (2). Hence, so far as the statistics 
of metastable states is concerned, the mean-field Hamiltonian Tid yields in fact a very 
close approximation to the pure Hamiltonian Ti. For the purpose of comparison we note 
that a = 0.1992 and a = 0.3552 for the binary ±1 2-spin glass 0, ^ and 4-spin glass 
models ITB, p9i, respectively, while a = ln2 0.6931 for the random energy model |11 



In Fig.0 we show the exponent a as a function of the energy density e. For the sake 
of clarity we present only the region of positive values of a. The lowest value of e at which 
the exponent a vanishes, denoted by eo, gives a lower bound to the ground-state energy 
density of the spin model defined by the Hamiltonian (Q) [|30[. We find eo = 0.0202845 
which, within the numerical precision, is exactly the value predicted by the first step of 
replica-symmetry breaking as well as by Golay's ergodicity hypothesis [0, This 
coincidence between the replica and the density of metastable states predictions for the 
ground-state energy occurs also in the random energy model [|Il|, |1^ . 

A similar study of the symmetrized version of the mean-field Hamiltonian (^, in 
which J,^,- = Jf^, yields exactly the same expression for the exponent a, see eq.(|13|). 



provided that the energy density e is replaced by 6^/2. Hence the symmetrization 
procedure results in a trivial rescaling the energy densities of the local minima, without 
affecting their number. 



3. Energy Barriers and Basin Sizes 

The picture that comes out of the replica approach to disordered spin models is that 
the phase space V composed of the 2^ spin configurations is broken into several valleys 
connected by saddle points p2[. The relative location and energetic properties of valleys 
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and saddles are expected to determine e.g. the ease with which the ground state can be 
reached. 

It will be convenient to introduce the notion of saddle-point energy E[s, w] between 
two (not necessarily strict) minima s and w. Denoting, for the sake of generality, the 
energy of state x by f{x), we can write 

= min |max [/(z)|z G p] p : path from s to w| , (14) 

where a path p is a sequence of configurations connected by one-spin flips (or, more 
generally, by moves taken from any desired "move set" ). The saddle-point energy E[s, w] 
forms an ultrametric distance measure on the set of local minima, see e.g. |3l|. The 
barrier enclosing a local minimum is the height of the lowest saddle point that gives 
access to an energetically more favorable minimum. In symbols: 

B{s) = min {E[s,w] - f{s)\w : f{w) < f{s)} (15) 

If B{s) = then the local minimum s is marginally stable. It is easy to check that 
eq.(|l5D is equivalent to the definition of the depth of local minimum in It agrees 
for metastable states with the more general definition of the depth of a "cycle" in the 
literature on inhomogeneous Markov chains |l|, |^, |^ . 

The information contained in the energy barriers is conveniently summarized by two 
global parameters that e.g. determine the convergence behavior of Simulated Annealing 
and related algorithms. The depth of a landscape [1^, ||, [1^, ^ is defined as 

D = max {B{s)\s is not a global minimum } . (16) 

It can be shown that Simulated Annealing converges almost surely to a ground state 



if and only if the cooling schedule satisfies X]fc>o ^^P(~'-^/-^fc) ~ ^ 113 • order 
to make the depth comparable between different landscapes we shall consider below 
the dimensionless parameter D/cr, where o"^ is the variance of the energy across the 
landscape. A related quantity is the (dimensionless) difficulty ^ of the landscape, 
defined by 

ip = max I -rr-. — ^-77 r s is not a global minimum ]■ (17) 

'-/(s)-/(mm) J 

where /(min) is the global energy minimum and the maximum is taken over non-global 
minima only. It is directly related to the optimal speed of convergence of Simulated 
Annealing. 

Since a direct evaluation of eq.([T^) would require the explicit constructions of all 
possible paths it does not provide a feasible algorithm for determining £^[5, w] even if N 
is small enough to allow an exhaustive survey of the landscape. The values of E[s,w] 
and B{s) can, however, be retrieved from the barrier tree of the landscape. Barrier trees 
have been considered recently in the context of RNA folding [|12[ and under the name 
"disconnectivity graphs" in the protein folding literature 0, In this contribution we 
use a modified version of the program barriers, which was developed for the analysis 
of RNA folding landscapes in [|12|. For the sake of completeness we briefly outline the 
definition of the barrier trees below. 
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Figure 2. Example of a barrier tree of a landscape. Data belong to a Gaussian REM 
with N = 7. The leaves 1-12 denote the local minima. The global minimum 1 is 
marked with an asterisk. Saddle points are labelled with capital letters from A to 
G. The saddle points B, C, D, E are "degenerate" indicating that the lowest energy 
paths leaving e.g. 4,5,8 run through a common exit point. (Note that all 2*" = 128 
configurations have pairwise distinct energies, hence there are no two distinct saddle 
points with the same energy, which may exist e.g. in the LABSP.) The Barrier of 5 is 
B(5) = E{D) - E{5), along the lowest path from 5 to 4, while 5(4) = E{E) - E{4:), 
along the lowest path from 4 to 1*. 



For simplicity let us assume that the energies of any two spin configurations are 
distinct, i.e., there is a unique ordering of the spin configurations by their energies. The 
construction of the barrier tree starts from an energy-sorted list of all configurations in 
the landscape. We will need two fists of valleys throughout the calculation: The global 
minimum x[l] belongs to the first active valley Vi, while the list of inactive valleys is 
empty initially. Going through this list of all configurations in the order of increasing 
energy we have three possibilities for the spin configuration x[k] at step k. 

(i) x[k] has neighbors in exactly one of the active valleys Vi. Then x[k] belong to Vi. 

(ii) x[k] has no neighbor in any of the (active or inactive) valleys that we have found 
so far. Then x[k] is a local minimum and determines a new active valley V/. In the 
barrier tree x[k] becomes a leaf. 

(iii) x[k] has a neighbor in more than one active valley, say V^j, . . . , V^^}. Then it is 
a saddle point connecting these active valleys. In the barrier tree x[k] becomes an 
internal node. In this case we add x[k] to valley V^j with the lowest energy. Then we 
copy the configurations ofVi^, . . . , Vi^ to V^^. Finally, the status of Vi^ through Vi^ is 
changed from active to inactive. This reflects the fact that from the point of view 
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of a configuration with an energy higher than the saddle point x[k], VJ^,...,1^^ 
appear as a single valley that is subdivided only at lower energy. Consequently, 
after the highest saddle-point energy has been encountered, all valleys except for 
the globally optimal Vi are in the inactive list. 

The outcome of this procedure is a tree such as the one shown in Fig. ^. The leaves 
correspond to the valleys of the landscape, while the interior nodes denote the saddle 
points. The tree contains the information on all local minima and their connecting 
saddle points. Indeed, saddle-point energies, and energy barriers can be immediately 
read off the barrier trees. 

A precise definition of valleys and saddle points in a landscape requires that we 
take into account the degeneracies in the energy function, i.e., the existence of distinct 
spin configurations with identical energies, in particular, the presence of neutrahty, 
where neighboring configurations have identical energies p6|. Degeneracies complicate 
the construction of the barrier tree, since the energy-sorting of the landscape is not 
unique any more. The simplest remedy is to use the same procedure as above starting 
from an arbitrary energy sorting. In this case the order of degenerate configurations 
in the list is arbitrary but fixed throughout the computation. Before proceeding to a 
configuration with strictly higher energy a simple clean-up step needs to be included in 
the tree-building algorithm: adjacent valleys with E[s, w] = f{s) = f{w) are joined to a 
single valley. Note that the resulting barrier tree may still contain distinct valleys with 
the same energy, as the examples in Fig. ^ show. The leaves of the barrier trees are in 
general valleys which may contain more than one degenerate local minimum. 

There is a clear visual difference between the barrier trees for LABSP and the mean- 
field approximation MF at the one hand, and the ±1 4-spin Hamiltonian and the REM on 
the other hand. The main difference appears to be a much larger amount of degeneracy 
in LABSP/MF, in particular highly degenerate ground states. In fact, it can be shown 
that the pure Hamiltoninan (|l|) has many nontrivial symmetries, besides the trivial one 
where x is replaced by — x, which are then responsible for the high degeneracy observed 
in the tree barrier . Obviously, the disordered Hamiltonian @) cannot have the same 
symmetries as the pure one, and so its high degeneracy stems simply from the extreme 
dilution of the couplings Jfy All models, except REM, are symmetric under replacing 
X by — x, hence all states appear in pairs. We note that the barrier tree of the ±1 
4-spin model is reminiscent of the "funnels" discussed e.g. in protein folding, with a 
large energy difference between the two global optima and almost all local "traps". In 
contrast, the REM shows, as expected, no relationship between energy and nearness of 
local minimum to the global one. 

During the construction of the barrier tree it is easy only to compute the lowest 
barrier B'{s) from s to a local minimum that comes earlier in the list of configurations, 
instead of the lowest barrier B{s) to a local minimum with strictly smaller energy. 
Clearly, B'{s) < B{s) since we take the minimum over a few more configurations 
than prescribed by eq.(|l5|). In case of degenerate landscapes our version of the 
barriers program calculates B'{s) which depends on the ordering of the degenerate 
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Figure 3. Tree representation of typical landscapes with N = 16. Upper left LABSP, 
upper right mean-field approximation, lower left integer 4-spin model, lower right 
Gaussian REM for N = 14 since for = 16 the number of minima is too large to 
allow a meaningful drawing. 

configurations. We obtain, however, B{s) = B'{s) for at least one of tlie valleys at each 
energy level. The fact that in eq. ([l6D we are required to maximize in particular over 
the barriers necessary to escape from any given energy level, however, implies that the 
values of depth and difficulty can be obtained directly from B'{s) instead of B{s). We 
note at this point that a modified version D' > D of the depth in which the maximum 
over all non-global minima is replaced by the maximum over all minima except one 
global minimum x* can also be obtained by the simplified procedure above, since it can 
be shown that D' is independent of the choice of x* 0. The parameter D' also appears 
in exact results on the convergence of Simulated Annealing. 
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Figure 4. Depth and Difficulty. Symbols: • LABSP, ■ mean-field approximation, A 
±l-version of the 2-spin model, ♦ ±l-version of the 4-spin model, ★ Gaussian REM. 
Data are averaged over 100 instances (50 instances for N = 20); the error bars show 
the width of the distribution, not the standard error of the means. 



Depth and Difficulty are shown in Figure ^ as a function of the number N of 
spins. While there are (moderate) quantitative differences, there does not seem to 
be a qualitative difference between the LABSP, the mean-field approximation, and the 
discretized 2 and 4-spin models. Note that all landscapes with the exception of the 
Gaussian REM have constant scaled depth D/a, while D/a increases linearly with the 
system size in the REM. 

A linear regression of the difficulties yields the slopes 0.595 ± 6, 0.926 ± 14, and 
0.07 ±2 for the mean-field Hamiltonian, the ±l-version of the 4-spin model and the ±1 
2-spin model, respectively. As expected, the difficulty of the quadratic model is much 
smaller than the difficulty of the 4-spin model. For the sake of clarity, we have omitted 
the data about the mean difficulty of the REM since it is too large as compared to those 
shown in the figure. Moreover the width of its difficulty distribution is also so large that 
the mean value is not physically meaningful. 

Additional information on the local minima can be traced during the construction of 
the barrier tree. We say that a configuration x belongs to the basin of the local minimum 
s if s is the endpoint of the gradient walk (steepest descent) starting in x. (Recall that 
each step of a gradient walk goes to the lowest energy neighbor.) By determining the 
valley to which the lowest energy neighbor of x[k] belongs we may for instance record the 
basin size of each local minimum. In a landscape without neutrality the gradient walk is 
uniquely determined by the initial condition, hence the basins form a partition of the set 
of configurations. We neglect the effects of neutrality in our numerical data by directing 
the gradient walk to the first possibility in the energy-sorted list of configurations. 

Computationally we find, for all models but the REM, that there is an approximate 
linear relationship between the energy of a local minimum and the logarithm of the 
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Figure 5. The logarithm of the size of the gradient- wallc basin of attraction of the 
minima as function of their energies for = 20. The data for MF and the ±1 4-spin 
model are superpositions of 10 instances, while for the REM we show only a single 
instance. 



size of its gradient-walk basin of attraction, see Figure |^. The fact that the deepest 
valleys have the largest basins of attraction can be understood as a consequence of the 
correlation between neighboring spin configurations in all landscapes with the exception 
of the REM, for which all low-energy minima have essentially the same size of basin of 
attraction. 



4. Discussion 

The performance evaluation of local search heuristics, in particular Simulated Annealing, 
in typical instances of optimization problems is a relatively new subject, where the 
existing criteria for measuring the hardness or difficulty of a problem are still not widely 
known or accepted, as compared to e.g. the more traditional worst-case analysis. In 
fact, on the one hand, one expects that the average number of local minima may serve 
as a measure of the problem hardness, while, on the other hand, one must concede 
that only local minima separated by high energy barriers are potential traps for the 
search heuristic. In this paper we combine the concepts of depth and difficulty from the 
Simulated Annealing literature to the average density of states calculations from the 
statistical mechanics of disordered systems to obtain a reasonably complete statistical 
description of the energy landscapes of several classic disordered models. The motivation 
is to compare the statistical features of these landscapes with the properties of a rather 
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puzzling deterministic problem — the low autocorrelated binary string problem (LABSP) 
— which has been identified as a particularly hard optimization problem for search 
heuristics such as Simulated Annealing. 

Our results indicate that there is only a quantitative difference between the 
depths and difficulties, as defined in the Simulated Annealing literature, of all models 
investigated, with the exception of the random energy model (REM) for which the 
complete lack of correlations between the energies of neighboring configurations results 
in a genuine golf-course type landscape. Hence, we have found no evidences of a golf- 
course like structure in the LABSP landscape, which resembles much more a correlated 
spin-glass model than the REM. It must be emphasized that although the pure LABSP 
model (0) may have a glass phase characterized by uncorrelated equilibrium states (at 
least its mean- field version has such a phase 0), the mere existence of this phase is no 
evidence of a golf-course like structure which, as mentioned above, requires vanishing 
correlations between the energy values of neighboring spin configurations. 

Perhaps the "golf-course" conjecture ^ stems simply from the fact that for 
large N the LABSP is a much more difficult problem for Simulated Annealing than the 
familiar quadratic spin glass, as shown in Fig. ^. Interestingly, the pairwise comparison 
between the problems indicates that those problems with the larger number of local 
minima have also the larger difficulty, the only exception being LABSP and the 4-spin 
glass model. It should therefore be interesting to use these two problems as a test-bed 
for validating the hardness criteria proposed in the Simulated Annealing literature. 

Finally, our analysis has shown that the disordered, mean-field Hamiltonian Tid, 
eq.(0), describes surprisingly well the qualitative (e.g. the barrier trees) as well as the 
quantitative (e.g. number and typical energy of local minima) features of the pure model 
H, eq.(g). 
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